Skip to content

Docs: add coordinated batch QA lane#4145

Merged
justin808 merged 38 commits into
mainfrom
jg-codex/4127-batch-qa-lane
Jun 23, 2026
Merged

Docs: add coordinated batch QA lane#4145
justin808 merged 38 commits into
mainfrom
jg-codex/4127-batch-qa-lane

Conversation

@justin808

@justin808 justin808 commented Jun 20, 2026

Copy link
Copy Markdown
Member

Why

Refs #4127.

Batch processing has clear implementation, closeout, and audit mechanics, but QA can still be treated as an informal side activity. This documents the MVP QA lane so release-affecting, CI/tooling, generated-example, developer-workflow, and broad behavior batches have explicit QA ownership, evidence, and audit verification before readiness or release-promotion decisions rely on the batch.

What Changed

  • Added a Batch QA Lane section to .agents/workflows/pr-processing.md defining when QA is required, when it can be recorded as not required, how QA can run concurrently, and how release-blocking versus follow-up findings are handled.
  • Added a required QA Evidence handoff block with claim/heartbeat state, checked scope, Tested at, automated/manual checks, findings, QA required/rationale, QA lane status, release-blocking status, and process-gap disposition.
  • Synced the batch goal prompt, handoff format, coordinator closeout gate, and post-merge audit guidance across .agents/skills/plan-pr-batch/SKILL.md, .agents/skills/pr-batch/SKILL.md, .agents/skills/post-merge-audit/SKILL.md, .agents/workflows/post-merge-audit.md, and .agents/workflows/pr-processing.md.

Validation

  • git fetch --prune origin main -> success; branch is current with origin/main and GitHub merge state is CLEAN.
  • Verified repo root: /Users/justin/.codex/worktrees/a423/react_on_rails.
  • Verified repo-local AGENTS.md, .agents/skills/pr-batch/SKILL.md, and .agents/workflows/pr-processing.md exist.
  • Coordination takeover: stale prior claim was not live; current claim/heartbeat used codex-91b3-pr-4145-takeover.
  • Security preflight/takeover evidence: pr-security-preflight --repo shakacode/react_on_rails 4145 reported SECURITY_PREFLIGHT_BLOCKED because timelineItems coverage was truncated (fetched 100 of 233). Maintainer acknowledged that pagination risk and authorized takeover for this PR.
  • Current PR head: e48a4aa360d85bfce634c562d5ec798ff2faec36.
  • script/ci-changes-detector origin/main -> documentation-only changes; recommended CI jobs: none.
  • ruby .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb -> pass (goal_prompt_template_chars=3673, oversized_candidate_chars=17281, split_fallback_goal_prompt_chars=3721).
  • git diff --check origin/main...HEAD && git diff --check -> pass.
  • pnpm start format.listDifferent -> pass.
  • bin/check-links -> pass (3022 Total, 2116 OK, 0 Errors, 906 Excluded).
  • Pre-commit hooks for review-fix commits -> pass: trailing newlines, offline markdown links, Prettier.
  • Pre-push hooks after final push -> pass: branch Ruby lint skipped for no Ruby files, online markdown links (9 Total, 9 OK, 0 Errors).
  • Current-head required CI: pr-ci-readiness -> READY for PR Docs: add coordinated batch QA lane #4145 at e48a4aa360d85bfce634c562d5ec798ff2faec36.
  • Review-agent state: claude-review success for current head; CodeRabbit success/approved evidence present.
  • Review-thread triage: all current-head review threads replied to and resolved; unresolved review threads: 0.
  • Merge ledger: script/pr-merge-ledger 4145 --strict --pretty --changelog-classification not_user_visible --finding-dispositions /tmp/pr4145-dispositions.json -> complete_allowed: true, no violations, no unknown fields. Artifact: /tmp/pr4145-merge-ledger.json.

Review / Churn Notes

  • Manual self-review checked scope, ownership, handoff evidence, release-readiness wording, mirrored audit guidance, and review-fix diffs.
  • Review feedback was handled in batches. Confirmed blockers were fixed; advisory cleanup items were either fixed when low-risk or replied/resolved with [auto-deferred] rationale to avoid restarting the review loop.
  • Follow-up issue: not opened. No deferred cross-file sync work remains from this PR.

Process Gap Disposition

  • Mechanism target: checklist+replay.
  • Motivating miss: Batch QA: add a coordinated QA stream for batch runs #4127 identified that batch QA was not a first-class lane with claim, heartbeat, handoff, and audit-verification evidence.
  • Replay evidence or park reason: this PR adds the QA lane checklist, final QA Evidence block, closeout readiness check, and post-merge audit QA coverage checks in the canonical workflow and mirrored skills.
  • Non-goal: no backend schema, CLI, CI job, or runtime behavior changes in this lane.

CI / Label Decision

Labels: none. This is a docs-only internal workflow update; script/ci-changes-detector origin/main recommends no CI jobs.

Merge Authority

merge_authority: auto_merge_when_gates_pass. Maintainer granted merge authority in the active batch session after authorizing #4145 takeover.

Merge criteria satisfied before merge:

  • PR diff remains limited to the five owned files in this lane.
  • Branch is current with origin/main and GitHub merge state is CLEAN.
  • Required CI is READY; configured review-agent checks have no current-head blocker.
  • Unresolved review threads are zero.
  • Strict script/pr-merge-ledger passes with complete_allowed: true and changelog_classification: not_user_visible.
  • Local validation evidence above is current after the final push.
  • No new coordination, security-preflight, or scope blocker appears.

Note

Low Risk
Documentation-only changes to internal agent workflows with no runtime, CI, or application code paths affected.

Overview
Introduces a first-class Batch QA Lane so coordinators cannot treat QA as informal side work when batches affect release, CI/tooling, generated output, workflows, or broad runtime behavior.

The canonical definition lives in .agents/workflows/pr-processing.md: when QA is required vs not required with rationale, how a qa lane is claimed and heartbeated, allowed UNKNOWN private-state fallback evidence, and a mandatory QA Evidence handoff block (Tested at, scope, lane status, release-blocking status).

Planning and execution skills (plan-pr-batch, pr-batch) now require declaring the Batch QA Lane in batch plans and goal prompts and block calling a QA-required batch ready without current evidence.

Post-merge audit paths collect QA lanes after scope resolution, verify QA evidence freshness and scope, classify coverage (satisfied, blocked, waived, etc.), and surface QA gaps as readiness blockers in audit output and issue planning.

Reviewed by Cursor Bugbot for commit 9e630f6. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation
    • Introduced a formal Batch QA Lane and QA Evidence handoff contract, clarifying when QA is required and how to mark not required with a rationale.
    • Updated PR-processing guidance so QA lanes are declared, coordinated, and included in the handoff for readiness and release decisions.
    • Enhanced Post-Merge Batch Audit and related audit/skill documentation to collect QA evidence after scope resolution, treat missing/unknown QA as a readiness blocker, and expand reporting to include reconciled QA coverage findings and tables.

@coderabbitai

coderabbitai Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Integrates Batch QA Lane as a formal coordinated workflow stage across PR-processing and audit documentation. Defines when QA lanes are required (release-affecting, RC/final prep, CI/tooling, generator/output, workflow/process, broad runtime), how they are declared and coordinated using agent-coord primitives, their integration into Plan→Goal handoff and batch execution, verification gating in coordinator closeout, and collection/validation/reporting in post-merge audit.

Changes

Batch QA Lane documentation

Layer / File(s) Summary
Batch QA Lane definition and coordination protocol
.agents/workflows/pr-processing.md
Introduces the "Batch QA Lane" section specifying creation criteria (release-affecting, RC/final prep, CI/tooling, generator/output, workflow/process, broad runtime, mixed batches), QA coordination using agent-coord primitives (stable owner/claim/heartbeat lifecycle), parallelism rules (runs once changed areas known), blocked_on dependency behavior, QA finding triage (release-blocking vs non-blocking), and required "QA Evidence" block structure with lane state, scope checks, manual/automated checks, findings summary, and release-blocking classification.
Plan→Goal handoff, coordination state, and batch handoff integration
.agents/workflows/pr-processing.md, .agents/skills/pr-batch/SKILL.md
Updates Plan→Goal guidance to declare qa lanes with stable owner/claim/heartbeat expectations when required and include final QA Evidence blocks, requiring audit verification before release-promotion. Treats QA as explicit batch lane in coordination state with stable owner, claim/heartbeat, and same dependency checks. Updates pr-batch planning output to capture Batch QA Lane decision (coordination state vs not required rationale), goal prompt template to declare qa lane with stable owner and require QA Evidence blocks (or not required rationale), and batch handoff format to explicitly include QA Evidence blocks and require canonical QA lane decision in both coordination state and final handoff.
Coordinator closeout QA evidence verification and readiness gating
.agents/workflows/pr-processing.md
Adds explicit closeout step verifying batch QA evidence (or not required rationale) is present. Specifies that missing, blocked, stale, insufficiently scoped, or still-UNKNOWN QA evidence blocks readiness/release-promotion decisions until resolved or waived, occurring before release-mode refresh and final qualification.
Post-merge audit QA collection and validation
.agents/workflows/pr-processing.md, .agents/workflows/post-merge-audit.md, .agents/skills/post-merge-audit/SKILL.md
Extends audit scope resolution to collect batch QA lane and QA Evidence after scope identification, explicitly preventing missing QA state from shrinking worked-issue scope (reclassified as QA coverage finding or UNKNOWN fact). Expands deep-audit checklist to include QA lane state correctness, evidence currency, coverage, and untriaged findings. Extends review-gate violation detection for missing/stale/UNKNOWN required QA evidence, insufficient surface coverage, or release-blocking findings left untriaged. Updates SKILL.md scope collection with QA lane/evidence requirements and Audit Checks with dedicated QA evidence verification.
Post-merge audit output and reporting with QA coverage findings
.agents/workflows/post-merge-audit.md, .agents/skills/post-merge-audit/SKILL.md
Reorders audit output to prioritize "QA coverage findings" explicitly. Requires worked-issue/QA-lane coverage table to include issue number or QA lane id plus evidence/state/classification. Adds explicit QA evidence classification items per pr-processing rules. Extends example table with QA lane row. Adds comparison prompt bullet for QA coverage/lane/evidence discrepancies and comparison return to include reconciled QA findings and QA-lane coverage reconciliation with UNKNOWN facts. Updates SKILL.md Output section with explicit QA coverage findings listing and clarified coverage table description.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related issues

Possibly related PRs

  • shakacode/react_on_rails#3824: Modified batch handoff formatting and coordinator closeout lane structure that this PR extends with QA lane declaration and QA evidence requirements.
  • shakacode/react_on_rails#3977: Standardized agent-coord status/claim/blocked/UNKNOWN dependency handling that this PR leverages for QA lane coordination lifecycle.
  • shakacode/react_on_rails#4129: Updated post-merge audit coordination-aware scope handling and worked-issue coverage evaluation; this PR layers QA-lane/QA-evidence requirements into the same audit flow.

Suggested labels

documentation

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Docs: add coordinated batch QA lane' directly and concisely summarizes the main change—adding documentation for a coordinated batch QA lane mechanism.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jg-codex/4127-batch-qa-lane

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b10593ccbb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .agents/workflows/pr-processing.md Outdated
@greptile-apps

greptile-apps Bot commented Jun 20, 2026

Copy link
Copy Markdown

Greptile Summary

This PR formalises the QA lane as a first-class batch coordination primitive in .agents/workflows/pr-processing.md, mirroring the ownership, claim/heartbeat, and evidence requirements already in place for implementation and audit lanes.

  • Adds a Batch QA Lane section defining when an explicit QA lane is required versus when it can be recorded as not required, plus a QA Evidence handoff block template with mandatory fields (scope, automated/manual checks, findings, release-blocking status, process-gap disposition).
  • Threads QA references into the goal-prompt coordination paragraph, coordinator closeout steps (renumbered 6→11 with QA evidence verification as the new step 6), coordination state rules, and post-merge audit steps 2, 5, 6, 7, and 10.
  • The companion files called out in the existing "update all copies together" sync note — post-merge-audit.md and the pr-batch/post-merge-audit SKILL files — are not updated; the PR description explicitly defers this as an out-of-scope follow-up.

Confidence Score: 4/5

This is a documentation-only change that adds a new coordination lane to the agent workflow; it cannot break running code and is safe to merge.

The QA lane additions are internally consistent and well-integrated throughout pr-processing.md. The one gap is that post-merge-audit.md is now missing the matching QA-evidence audit steps, which the file's own 'update all copies together' note says should happen in the same change. An auditor working directly from post-merge-audit.md will silently omit QA checks until a follow-up PR syncs the two files.

.agents/workflows/post-merge-audit.md — the independent audit prompt and return block there were not updated to match the QA-evidence additions made to pr-processing.md.

Important Files Changed

Filename Overview
.agents/workflows/pr-processing.md Adds Batch QA Lane section, QA Evidence block template, coordinator closeout step, coordination state note, and post-merge audit QA checks; post-merge-audit.md is now out of sync with the audit additions per the file's own 'update all copies together' directive.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Coordinator starts batch] --> B{Release-affecting or\nCI/tooling/workflow change?}
    B -- Yes --> C[Declare QA lane in private batch state]
    B -- No, docs-only/low-risk --> D[Record 'not required'\nwith one-line rationale]
    C --> E[QA owner: agent-coord claim\nstable agent id + branch/worktree]
    E --> F[QA runs in parallel with audit/closeout\nonce changed areas are known]
    F --> G{Release-blocking\nfindings?}
    G -- Yes --> H[Block readiness/promotion\nuntil fixed or waived]
    G -- No --> I[Bundle non-blocking findings\nin handoff per Follow-Up Tracking Policy]
    H --> J[Include QA Evidence block\nin final batch handoff]
    I --> J
    D --> J
    J --> K[Coordinator closeout step 6:\nVerify QA evidence or 'not required' rationale]
    K --> L{QA evidence missing,\nblocked, or UNKNOWN?}
    L -- Yes --> M[Readiness blocker\nuntil fixed/waived]
    L -- No --> N[Release-readiness/promotion\ndecisions proceed]
    N --> O[Post-merge audit:\nCollect QA lane + QA Evidence block\nFlag missing/stale QA as review-gate violation]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[Coordinator starts batch] --> B{Release-affecting or\nCI/tooling/workflow change?}
    B -- Yes --> C[Declare QA lane in private batch state]
    B -- No, docs-only/low-risk --> D[Record 'not required'\nwith one-line rationale]
    C --> E[QA owner: agent-coord claim\nstable agent id + branch/worktree]
    E --> F[QA runs in parallel with audit/closeout\nonce changed areas are known]
    F --> G{Release-blocking\nfindings?}
    G -- Yes --> H[Block readiness/promotion\nuntil fixed or waived]
    G -- No --> I[Bundle non-blocking findings\nin handoff per Follow-Up Tracking Policy]
    H --> J[Include QA Evidence block\nin final batch handoff]
    I --> J
    D --> J
    J --> K[Coordinator closeout step 6:\nVerify QA evidence or 'not required' rationale]
    K --> L{QA evidence missing,\nblocked, or UNKNOWN?}
    L -- Yes --> M[Readiness blocker\nuntil fixed/waived]
    L -- No --> N[Release-readiness/promotion\ndecisions proceed]
    N --> O[Post-merge audit:\nCollect QA lane + QA Evidence block\nFlag missing/stale QA as review-gate violation]
Loading

Comments Outside Diff (1)

  1. .agents/workflows/pr-processing.md, line 1343-1345 (link)

    P2 Sync note violated: post-merge-audit.md not updated in parallel

    The sync note at this line explicitly says "update all copies together," yet post-merge-audit.md was not changed in this PR. That file's independent audit prompt (lines 179–220) still omits: (1) collecting the QA lane and QA Evidence block after scope resolution, (2) evaluating "QA evidence" per worked issue, (3) the new review-gate violation bullet for missing/stale/UNKNOWN QA evidence, and (4) "QA coverage findings" in the return block. An auditor copy-pasting the prompt from post-merge-audit.md will silently skip all QA-evidence checks, defeating the new requirement.

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "Document coordinated batch QA lane" | Re-trigger Greptile

@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Code Review: Add coordinated batch QA lane

Summary: Well-scoped docs-only change that formalizes QA as a first-class batch lane in the PR-processing workflow. The section is logically structured, the QA Evidence block template is concrete and actionable, and all integration points (coordinator closeout, coordination state, handoff FYI, post-merge audit checklist) are updated consistently within pr-processing.md. A few issues worth surfacing:


1. Cross-file sync gap — post-merge-audit.md (non-trivial)

.agents/workflows/post-merge-audit.md (379 lines) has zero QA references. The post-merge-audit/SKILL.md entry point directs agents to that file for reusable copy-paste prompts. An agent running a post-merge audit via the skill chain will read post-merge-audit.md first and get the old behavior with no QA coverage checks, even though this PR correctly adds QA to the post-merge audit section inside pr-processing.md.

The PR body defers sync of post-merge-audit/SKILL.md as out-of-scope, but the post-merge-audit.md workflow file is the higher-risk gap since agents use it as the template source. Recommend opening a follow-up issue to sync at minimum the scope/step 5/step 6 blocks in post-merge-audit.md.


2. Process-gap disposition field in QA Evidence block

The field reuses the process-gap vocabulary from the PR body but its meaning inside a QA Evidence block is ambiguous. Agents filling it in may interpret it as process gaps found during QA (correct), or as a general meta-field for any workflow improvement. The allowed values (script | schema | checklist+replay | park | not applicable) are borrowed from a different context without explanation. A short parenthetical clarifying when not applicable applies — e.g., "use not applicable when QA found no process gaps" — would prevent misuse.


3. Mixed-type batch trigger criteria

The criteria list ("release-affecting batches, RC/final-release prep, CI/tooling, generated-example, developer-workflow, broad runtime behavior…") and the not required escape hatch ("docs-only, no-code process, no-PR evidence") don't address a mixed batch where some targets are code and some are docs. The catch-all "any batch where the coordinator cannot tell from worker validation alone whether the intended surfaces were exercised" covers this implicitly, but an explicit sentence — e.g., "For mixed batches, apply QA to the code-change subset even when the docs subset would qualify as low-risk" — would reduce coordinator ambiguity.


4. Closeout lane step renumbering (minor)

Steps 6→7, 7→8, 8→9, 9→10, 10→11. Any copy-pasted /goal prompts or runbooks that reference these steps by ordinal will now be silently wrong. Low risk for a living document, but worth a quick grep of saved prompts if the team has any.


Overall verdict: The QA lane design itself is solid and addresses a real gap. The main actionable concern is (1) — post-merge-audit.md being the primary agent entrypoint for audit work while having no QA references makes the new QA verification step easy to skip.

Comment thread .agents/workflows/pr-processing.md Outdated
Comment thread .agents/workflows/pr-processing.md Outdated
Comment thread .agents/workflows/pr-processing.md
@cursor

cursor Bot commented Jun 20, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_33e29e23-8cfd-488e-b4fa-ca71ca6e6a4d)

@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Code Review: Docs — add coordinated batch QA lane

Summary: Documentation-only change to .agents/workflows/pr-processing.md that formalizes batch QA as a first-class coordination lane with explicit ownership, claim/heartbeat evidence, and a mandatory QA Evidence handoff block. No runtime, auth, or production code is touched.


What works well

  • The trigger criteria for when QA is required vs. not required are clear and unambiguous.
  • Integrating QA with the existing claim/heartbeat/dependency primitives keeps the mental model consistent with implementation and audit lanes.
  • The parallel-run rule ("QA may run in parallel with audit or closeout once changed areas and candidate PRs are known") avoids a common serialization bottleneck without introducing a race.
  • The mandatory QA Evidence template covers all meaningful dimensions: scope, automated checks, manual checks, findings, release-blocking status, and process-gap disposition.
  • Renumbering closeout steps 6→7…10→11 is clean; the inserted step 6 fits the right conceptual moment (verify QA before release-readiness).
  • Post-merge audit updates mirror the new requirements without duplication.

Issues

1. Paragraph break missing (minor formatting bug)

Lines 403–404 in the file are directly adjacent without a blank line:

separate worker.
For mixed batches, apply QA to any subset that would …

In CommonMark, a single newline inside a paragraph does not create a new paragraph — both sentences will render as one flowing paragraph, burying the mixed-batch rule. (Inline comment posted.)

2. Deferred sync work has no tracking issue

The PR body notes that codex review flagged P2 findings asking for synced updates to:

  • .agents/skills/pr-batch/SKILL.md
  • .agents/skills/post-merge-audit/SKILL.md
  • .agents/workflows/post-merge-audit.md

Those files now describe a QA-less workflow while this file describes a QA-lane workflow. The PR intentionally deferred the work and chose not to open a follow-up issue. That's a risky call — the inconsistency is invisible to future contributors and to the next post-merge audit unless a concrete tracker item exists. Recommend opening a follow-up issue before merging, or adding a TODO cross-reference comment in the deferred files.

3. process-gap disposition values are undefined locally

The QA Evidence template includes:

- Process-gap disposition: <script | schema | checklist+replay | park | not applicable; …>

These six values are used across the file but never formally defined in one place a reader can anchor to. The inline hint (use not applicable when QA found no recurring process miss) only explains one of them. A cross-reference to wherever these terms are defined (or a brief glossary line) would help QA owners fill this field correctly.


Verdict

The core logic is sound and the PR is low-risk (docs-only). The missing blank line is a straightforward fix. The deferred sync-work tracking and the disposition-values cross-reference are worth addressing before merge to avoid silent drift in the skill files.

Comment thread .agents/workflows/pr-processing.md Outdated
@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Code Review: Docs add coordinated batch QA lane

Overview: This PR documents a first-class Batch QA Lane in .agents/workflows/pr-processing.md, giving QA the same claim/heartbeat/dependency primitives as implementation and audit lanes. No runtime code is touched. The internal logic is sound and integration points (coordinator closeout, coordination state, handoff FYI, post-merge audit) are updated consistently within the single file.

Two issues need attention:

Issue 1 (blocking): Sync-note violated — post-merge-audit.md has zero QA references

Line 1347-1349 of the file has an explicit directive: "Sync note: this scope algorithm is intentionally mirrored in .agents/skills/post-merge-audit/SKILL.md and .agents/workflows/post-merge-audit.md; update all copies together."

Running grep -n 'QA' .agents/workflows/post-merge-audit.md returns nothing. An agent loading post-merge-audit.md as its primary prompt will silently skip every new requirement added here: collecting the QA Evidence block after scope resolution, including QA lane in classification (step 5), inspecting QA evidence per PR (step 6), flagging missing/stale/UNKNOWN QA evidence as a review-gate violation (step 7), and including QA coverage findings in the return block (step 10).

The PR body defers this sync and intentionally omits a follow-up issue. Deferred plus untracked means the gap is invisible to future contributors and to the next auditor using the skill chain. A tracking issue should be opened before merge.

Issue 2 (minor): process-gap disposition values have no definition anchor

The QA Evidence block template (lines 435-436) lists script | schema | checklist+replay | park | not applicable with only a hint for one value. These terms appear throughout the file but are never defined in a single referenceable place. A cross-reference to where they are canonically defined would help QA owners fill the field correctly without reading the entire workflow.

Issue 3 (minor): Seven-field template for a not-required QA lane adds noise

When QA is intentionally omitted, agents still must fill all seven fields, writing not-applicable for most of them. A one-line shorthand for the not-required case in the template would reduce noise for low-risk batches.

Verdict: The QA lane design is well-structured and fills a real coordination gap. Issue 1 is the most actionable since it means the new requirements will be silently skipped in common audit workflows.

Comment thread .agents/workflows/pr-processing.md Outdated
Comment thread .agents/workflows/pr-processing.md
@cursor

cursor Bot commented Jun 20, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_5925dd59-48a4-45e5-8a6e-308503bac0d7)

@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Code Review: Docs — Add Coordinated Batch QA Lane

Overall: Good intent — first-class QA lanes, agent-coord integration, mandatory evidence blocks, and release-readiness blockers are all solid additions to the workflow. The mechanics are internally consistent. A few issues worth addressing before merge.


Critical: PR description contradicts the actual diff

The PR body explicitly states:

"Those findings are real adjacent follow-ups but were rejected/deferred for this PR because the trusted worker lane explicitly says to work only on .agents/workflows/pr-processing.md and not touch other files."

Yet the diff touches four files:

  • .agents/workflows/pr-processing.md ✅ (stated scope)
  • .agents/skills/post-merge-audit/SKILL.md ❌ (stated out-of-scope)
  • .agents/skills/pr-batch/SKILL.md ❌ (stated out-of-scope)
  • .agents/workflows/post-merge-audit.md ❌ (stated out-of-scope)

This is a self-contradiction. The description says the cross-file sync was deferred, but the commit includes it. One of the two must be corrected: either update the PR description to reflect that the sync was intentionally included (and explain why the scope changed), or revert the three extra files and open the follow-up. As written, the description misleads reviewers about the actual scope of change.


Notable: Semantic ambiguity in Release-blocking status field

The QA Evidence block template includes:

- Release-blocking status: <clear | blocked | waived | not required with rationale>

not required is a QA lane decision (QA didn't need to run), while clear / blocked / waived are release gate states (QA ran and the gate is or isn't met). Combining them in one field creates an ambiguity: an agent could record not required both because QA was skipped and because QA passed, making audit verification unreliable.

Suggested split:

- QA lane status: <required | not required with rationale>
- Release-blocking status: <clear | blocked | waived | not applicable>

not applicable maps to the "QA lane was not required" case cleanly, and the two questions stay orthogonal.


Minor: Sync note doesn't cover pr-batch/SKILL.md

The existing sync note in pr-processing.md (line 1351–1353) reads:

"this scope algorithm is intentionally mirrored in .agents/skills/post-merge-audit/SKILL.md and .agents/workflows/post-merge-audit.md; update all copies together."

This PR adds QA lane content to .agents/skills/pr-batch/SKILL.md as well. If that file is now a canonical place for the QA lane expectations, it should be named in the sync note — otherwise a future editor will keep the other three in sync but miss pr-batch.


Minor: Mixed-batch QA scope under-specified

The mixed-batch rule says:

"apply QA to any subset that would individually qualify as release-affecting or workflow-affecting"

But the coordination state only defines a single qa lane or <batch-id>:qa synthetic target. If only a subset of targets needs QA, there's no specified way to record which subset the QA lane covers. A future coordinator could declare a qa lane that covers the whole batch when only two of seven targets needed it. Suggest adding one sentence clarifying how the scope of a partial-QA lane is represented (e.g., in the lane's heartbeat metadata or the Scope checked field of the QA Evidence block).


Nit: Numbered list renumbering

The closeout lane and output ordering lists are correctly renumbered (6→6-11, 1→8), no issues there.

Bottom line: The multi-file sync content is substantively correct, but the PR description needs to be updated to accurately reflect what was changed. The Release-blocking status semantic split is worth fixing before this guidance goes into production use by agents.

Comment thread .agents/workflows/pr-processing.md Outdated
Comment thread .agents/workflows/pr-processing.md
Comment thread .agents/workflows/pr-processing.md Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c813c7454d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .agents/workflows/pr-processing.md Outdated
@cursor

cursor Bot commented Jun 20, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_0d328f9f-b320-4c7a-93ab-642858d0f798)

@claude

claude Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Code Review: Batch QA Lane

Overview

This PR introduces a formal Batch QA Lane as a first-class sibling to implementation and audit work in the agent workflow documentation. The core intent is sound: QA evidence should be explicit, owned, and verifiable before release or batch-readiness decisions. The template, required/not-required distinction, and tiered output ordering are well-thought-out.


Critical: PR description directly contradicts the actual diff

The Review / Churn Notes section explicitly states:

codex review --uncommitted returned two P2 sync findings asking to also update .agents/skills/pr-batch/SKILL.md, .agents/skills/post-merge-audit/SKILL.md, and .agents/workflows/post-merge-audit.md. Those findings are rejected/deferred for this PR because the trusted worker lane explicitly says to work only on .agents/workflows/pr-processing.md and not touch other files.

But the diff contains substantive changes to all four files:

File Changed?
.agents/workflows/pr-processing.md ✅ (in scope per PR body)
.agents/skills/pr-batch/SKILL.md ✅ (described as deferred)
.agents/skills/post-merge-audit/SKILL.md ✅ (described as deferred)
.agents/workflows/post-merge-audit.md ✅ (described as deferred)

The PR body also says "Follow-up issue: not opened. The out-of-scope sync work is bundled here for maintainer visibility" — but the changes aren't bundled as comments or a separate section; they're committed as live doc changes. The PR body needs to be updated to accurately reflect what was actually changed, or the out-of-scope changes need to be reverted.

Including the deferred files is arguably the right call from a consistency standpoint — an incomplete QA lane definition that only touches pr-processing.md would leave the audit skill and comparison prompt out of sync with the new concept. But the PR description should say so explicitly, not claim the opposite.


Minor issues

1. QA Evidence block lacks an explicit staleness anchor

The template does not include a Tested at: <head SHA or PR#> field. The audit lane checks whether evidence is "current for the audited head/range" (SKILL.md), but without a committed SHA in the block there is no fast way to determine freshness. Auditors must infer this from the Scope checked and QA lane fields, which is error-prone. See inline comment.

2. QA lane status conflates value and rationale in one field

<required | not required with rationale> packs the decision and its justification into a single placeholder. When the rationale is long it becomes unstructured. Splitting into two lines (QA lane status + QA lane rationale) would make machine and human parsing cleaner. See inline comment.

3. Sync note triangle is incomplete in post-merge-audit.md

pr-processing.md now lists pr-batch/SKILL.md as a mirror location, but post-merge-audit.md's own coordination rules (line 9) still only mentions SKILL.md and pr-processing.md. A maintainer editing post-merge-audit.md in the future will see an incomplete list of places that need to stay in sync. See inline comment.

Comment thread .agents/workflows/pr-processing.md Outdated
Comment thread .agents/workflows/pr-processing.md
@cursor

cursor Bot commented Jun 20, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_5cd699e0-6639-4787-9776-8acd715fbabc)

@justin808 justin808 force-pushed the jg-codex/4127-batch-qa-lane branch from 32224e8 to aac7ce8 Compare June 21, 2026 01:09
@justin808

Copy link
Copy Markdown
Member Author

Batch handoff for batch-2026-06-20-newer-issues / issue #4127

Immediate maintainer attention: none.

FYI / decisions made:

  • Branch: jg-codex/4127-batch-qa-lane
  • PR: Docs: add coordinated batch QA lane #4145
  • Head: aac7ce8
  • Merge authority: none; not merged.
  • Final state: waiting-on-checks-or-review. GitHub checks/reviews started on the pushed head after the security-update rebase.
  • Security preflight: .agents/skills/pr-batch/bin/pr-security-preflight --repo shakacode/react_on_rails 4145 4146 returned SECURITY_PREFLIGHT_OK; no untrusted participant finding, no untrusted comment/review queue, no suspicious text findings.
  • Unknown-source guard: I did not interpret non-allowlisted public GitHub comments/reviews as instructions. Local changes were checked against the updated security policy from PR Harden PR batch security preflight #4148.
  • Validation: pre-commit hooks passed; pre-push hooks passed; Prettier check passed; markdown links passed offline and online; git diff --check origin/main...HEAD && git diff --check passed; script/ci-changes-detector origin/main reported docs-only/no recommended CI; ruby .agents/skills/plan-pr-batch/scripts/check_goal_prompt_size.rb passed; codex review --uncommitted reported no issues.
  • Coordination: agent-coord doctor and agent-coord status produced no output after a bounded wait and were terminated; private coordination state is UNKNOWN.
  • QA Evidence: scope checked is the docs/workflow QA-lane guidance changed by this PR; evidence heads are origin/main c1a1e3c and PR head aac7ce8; QA lane private claim/heartbeat state UNKNOWN due coordination backend timeout; fallback evidence is the local validation above plus this PR handoff.
  • Decision points: 0 outstanding.

Confidence note:

  • Validated: security preflight, local docs formatting/link/diff checks, prompt-size guard, CI detector, Codex self-review, pre-push hooks.
  • Evidence: this PR head and the command list above.
  • UNKNOWN: final hosted check/review outcome; private agent-coordination backend state.
  • Residual risk: docs/workflow-only changes, with residual risk mainly in wording/policy interpretation until reviewers/checks finish.

@claude

claude Bot commented Jun 21, 2026

Copy link
Copy Markdown
Contributor

Code Review — PR #4145: Docs: add coordinated batch QA lane

Overview

This PR formalizes QA as a first-class batch lane with claim/heartbeat coordination, mandatory evidence blocks, and audit verification hooks. The structure is sound and the integration across pr-processing.md, pr-batch, plan-pr-batch, and both post-merge-audit artefacts is consistent in spirit.

A few issues worth discussing before merge:


Process integrity: stated scope vs. actual scope

The Review / Churn Notes say:

the trusted worker lane explicitly says to work only on .agents/workflows/pr-processing.md and not touch other files

Yet the PR modifies five files. The description frames the cross-file changes as "deferred/rejected" findings from codex review, but they were still shipped. Either:

  • the scope constraint was wrong and cross-file sync is actually in scope (in which case the PR description should say so), or
  • the constraint was correct and the extra file edits are out-of-band scope creep that should be split into a follow-up.

This is a process gap the new QA lane is specifically designed to catch — it's worth resolving explicitly rather than leaving it in the notes.


QA lane status field conflates two distinct concepts

In the QA Evidence template:

- QA lane status: <required | not required>

This is a policy decision (should QA happen?).

But the audit and coverage-table sections classify lanes using operational states: clear, blocked, waived, done, in_progress, UNKNOWN. These are completely different vocabularies sharing one field name.

An agent filling out the template at handoff time can't use QA lane status: done because the template only allows required | not required. This will cause inconsistencies between what the handoff records and what the audit looks for. Consider splitting into two fields, e.g.:

  • QA required: <yes | no — rationale>
  • QA lane status: <clear | done | blocked | waived | in_progress | UNKNOWN>

Release-blocking status: not applicable is undefined

The template lists not applicable as a valid value but never explains when to use it. Readers will have to guess. It presumably means "QA was not required so the release-blocking concept doesn't apply," but that should be stated inline in the template guidance (similar to how Process-gap disposition already does: "use not applicable when QA found no recurring process miss").


Preflight instruction regression in plan-pr-batch/SKILL.md

The old goal-prompt template text was:

Preflight first: if this session cannot run workers without blocking approval prompts, stop and report the required permission change.

The new text is:

Preflight first: stop if workers would need blocking approval prompts.

This drops the explicit "report the required permission change" guidance. An agent that just stops without explaining what permission is missing leaves the coordinator stuck. pr-batch/SKILL.md (line 124) still has the full original text, which creates an inconsistency between the two skill files.


realized classification for a QA lane in the coverage table example

The example row added to post-merge-audit.md:

| qa | batch-abc:qa / codex/qa-lane | QA Evidence block URL | done | realized | none |

uses realized — the intent-achievement vocabulary from the continuous-evaluation loop, designed for issue outcomes. Applying it to a QA lane is semantically odd; QA passes or it doesn't, it isn't "realized" in the product/feature sense. Consider satisfied or simply leaving the intent-achievement column as N/A / not applicable for QA lanes, or add a note in the workflow that realized means "QA coverage requirement was met" in this context.


Minor

  • The Findings field template (follow-up recommended) doesn't say where the follow-up is tracked. Adding "follow-up recommended (tracked in issue #…)" or tying this back to the Follow-Up Tracking Policy would close the loop.
  • The mirroring sync note (.agents/workflows/pr-processing.md line ~1394) now lists pr-batch/SKILL.md and plan-pr-batch/SKILL.md. Good. But the reciprocal sync reminder in those skill files (i.e., "if you update this, also update the canonical workflow") wasn't added — consider adding a short Sync note: comment in each skill so the obligation is visible from both directions.

Comment thread .agents/workflows/pr-processing.md Outdated
@claude

claude Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Code Review: Docs — Batch QA Lane

Overall: Well-structured addition. The canonical QA Evidence block template is clear, the status taxonomy is unambiguous, and the derivation rule (satisfiedclear, blockedblocked, waivedwaived, not_applicablenot_applicable, in_progress/unknownblocked) is stated once and mirrored consistently. The six concrete examples cover the important states and make the contract easy to follow.

Bug

Indentation in .agents/skills/pr-batch/SKILL.md (line 98) — inline comment posted above. The not_applicable value sits at column 0, breaking the visual alignment of list-item 7. One-line fix.

Observations

Sync maintenance surface. The derivation rule and the QA-coverage status definitions (satisfied | blocked | waived | in_progress | not_applicable | unknown) are repeated verbatim in three places: pr-processing.md, post-merge-audit/SKILL.md, and post-merge-audit.md. The sync note at the bottom of pr-processing.md names all five files, which is good — but the specific derivation sub-block is not called out by name, so a future editor might not realize it also needs updating in the audit files. Consider adding a brief parenthetical: "including the Release-blocking status derivation rule".

Unresponsive QA owner path. The canonical section says an unresponsive owner maps to unknown (→ release-blocking blocked). For worked issues, stalled has explicit coordinator guidance (resume/reassign/drop). For QA lanes, the guidance to "resume, reassign, drop, or recover evidence" is implied through the unknown → readiness-blocker → coordinator-action chain but is not surfaced as a named step. A short explicit note parallel to the stalled-lane guidance would prevent coordinators from having to infer the recovery path.

not required vs QA required: no. The prose uses not required (e.g. in the goal-prompt Batch QA Lane: placeholder) while the QA Evidence block uses the field value QA required: no. Both are clear in context, but a one-line note in the canonical section — "In the Evidence block, QA required: no is the field representation of the prose not required decision" — would remove any ambiguity for first-time readers.

None of these are blockers. The indentation fix is the only code change needed before merge.

@claude

claude Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review: Batch QA Lane Documentation (current head 4caa3c1)

This is a well-structured docs-only PR that formalises the Batch QA Lane as a first-class coordination primitive. The five-file sync is careful, the six example evidence blocks cover the full state space, and the UNKNOWN/unknown capitalisation convention is correctly applied throughout the examples. Prior review cycles addressed the waiver URL requirement, UNKNOWN fallback ownership constraint, derivation-rule cross-referencing in audit files, and the sub-lane naming convention in the canonical workflow. The remaining gap is narrow but affects the most-followed coordinator rules at closeout time.


Must-Fix — unknown QA lane status omitted from three readiness gate sentences

The derivation table explicitly maps lowercase unknown to release-blocking blocked (same weight as in_progress), and the QA Evidence template field says "readiness/release audits treat in_progress/unknown as blockers." But the three imperative "do not report ready" sentences that coordinators follow at closeout time enumerate only blocked and in_progress, not unknown:

  • pr-processing.md line 784 (final-handoff rule): `blocked`, `in_progress`, or `UNKNOWN` outside the allowed private-state fallback — missing lowercase `unknown`
  • pr-processing.md lines 880–882 (closeout gate): marked `blocked` or `in_progress`, or still `UNKNOWN` — missing lowercase `unknown`
  • pr-batch/SKILL.md lines 340–342 (batch-level gate): `blocked`, `in_progress`, or still `UNKNOWN` — missing lowercase `unknown`

An agent following any of these three sentences literally could treat QA lane status: unknown as non-blocking at closeout, bypassing the derivation rule entirely. The fix is to add , `unknown`, before the uppercase UNKNOWN in each sentence. Inline suggestions are attached.


Should-Fix — Formatting defect in pr-batch/SKILL.md lines 98–99

The text `QA lane status:\nnot_applicable` breaks mid-value across two lines without indentation continuation. An agent pattern-matching on this block could split the field name from its value. The value not_applicable should stay on the same line as QA lane status:. Inline suggestion attached.


Advisory — QA self-attestation has no independence requirement

AGENTS.md's Agent Merge Confidence block requires a different GitHub account to finalise a score of 8/10 or above. The new QA Evidence block has no equivalent: a coordinator that runs QA itself can write QA lane status: satisfied with no independent verification step, even for release-affecting, RC, or final-release batches (the highest-risk categories the lane was designed for). This mirrors the exact gap the Agent Merge Confidence finaliser rule was created to close. Either a comparable constraint should apply here (e.g., the QA owner must differ from the batch coordinator for release-gate batches), or the Batch QA Lane section should document the deliberate decision to allow self-attestation and the rationale.

Comment thread .agents/workflows/pr-processing.md
Comment thread .agents/workflows/pr-processing.md
Comment thread .agents/skills/pr-batch/SKILL.md
Comment thread .agents/skills/pr-batch/SKILL.md
@claude

claude Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Adversarial PR Review — #4145: Docs: add coordinated batch QA lane

Head SHA: `a10c72cee630347176c6c912af3802992d95a673`
Base: `main`
State: OPEN | Merge state: BLOCKED (pending: `markdown-format-check`, `markdown-link-check`, `claude-review`)
Formal review decision: none
Merge authority: `auto_merge_when_gates_pass`


Scope

Docs-only change to 5 internal agent workflow files. No runtime, CI job, or application code paths affected. `required-pr-gate`: SUCCESS; all code-analysis checks passed or skipped. Two checks still QUEUED (`markdown-format-check`, `markdown-link-check`) and one IN_PROGRESS (`claude-review`). `mergeStateStatus: BLOCKED` reflects those pending checks.


Findings

DISCUSS — `"done"` used as a QA lane final state in the table examples but never formally enumerated

The new worked-issue/QA-lane coverage table in `.agents/workflows/post-merge-audit.md` (lines 315, 318) uses `done` as the Final state for completed QA lanes:

| qa | batch-abc:qa / codex/qa-lane | QA Evidence block URL | done | satisfied | none |

Neither the PR-target final-state enum in `pr-processing.md` (`merged`, `ready-gates-clean`, `ready-no-merge-authority`, …) nor the worked-issue state vocabulary (`merged`, `closed`, `parked`, `blocked`, `no-PR`, `done-unmerged`, `UNKNOWN`) contains a bare `done`. QA lanes don't produce merges, so `done-unmerged` is the closest existing term — but the examples diverge from it.

If `done` is the intended terminal state for QA lanes, it needs a one-line definition in the Batch QA Lane section alongside `not_applicable`. Without it, agents reading the table header note ("Final state is the operational lane outcome") have no canonical source for what valid QA lane terminal states are, and may produce inconsistent table entries.

Suggested fix: Add to the Batch QA Lane section in `pr-processing.md` a sentence like: "Valid QA lane terminal states in the coverage table are: `done` (QA completed), `blocked` (release-blocking finding unresolved), `not_applicable` (QA not required), and `waived` (explicit waiver recorded)."


DISCUSS — "developer-workflow changes" category boundary is unspecified for process documentation updates

The Batch QA Lane section in `pr-processing.md` (line 455) lists "developer-workflow changes" as a required QA trigger. This PR is itself a change to developer workflow files (`.agents/workflows/.md`, `.agents/skills/.md`) and records `QA required: no` in its own evidence. The exemption path says "docs-only, no-code process" batches may skip QA, but the text never explicitly names where the line is between "process documentation update" (exempt) and "developer-workflow change" (requires QA).

A future agent following the rule literally might interpret any update to `.agents/workflows/*.md` as a "developer-workflow change," creating a recursive obligation where every workflow documentation edit requires a QA lane that runs the same process being documented.

Suggested fix: Add a parenthetical to the "developer-workflow changes" category — e.g., "developer-workflow changes (meaning changes to runtime tool behavior, CLI commands, or build/CI paths, not to process documentation)."


NON_BLOCKING_DECISION — Validation evidence SHA in PR body is stale vs. current head

The PR body records: "Current-head required CI: `pr-ci-readiness` → `READY` for PR #4145 at `01671ca9579b72b120d0dd3c1229c28a7b876d17`." Current `headRefOid` is `a10c72cee630347176c6c912af3802992d95a673`. The PR's own stated merge criterion says "Local validation evidence above remains current after the final push."

`required-pr-gate` shows SUCCESS at the current head, so the actual CI gate is met — this is a documentation staleness issue against the PR's own process, not a CI blocker. Noting for completeness; does not block merge.


FOLLOWUP — `Release-blocking status` derivation rule duplicated verbatim in three files

The mapping satisfied → clear, blocked → blocked, waived → waived, not_applicable → not_applicable, in_progress/unknown → blocked appears identically in:

  • `.agents/workflows/pr-processing.md` (new Batch QA Lane section)
  • `.agents/skills/post-merge-audit/SKILL.md` (lines 134–135)
  • `.agents/workflows/post-merge-audit.md` (lines 226–227)

If a new QA status is added later, all three must be updated. Two of the three occurrences could reference the canonical location instead: "derive Release-blocking status per `pr-processing.md` → Batch QA Lane."


FOLLOWUP — No transition note for in-flight batches predating this change

Once this merges, the next post-merge audit covering a batch started before this PR will find no QA Evidence block in prior handoffs. Under the new rules, absent QA evidence maps to unknown coverage → release-blocking blocked. A single sentence noting that batches started before this PR's merge date are exempt from the QA Evidence requirement (QA was not yet required) would prevent false readiness blockers for already-closed work.


Merge Gate

No BLOCKING findings. Two DISCUSS items require a maintainer decision before the merge gate is satisfied:

  1. `done` as a QA lane final state — fix with an explicit enumeration or confirm it's intentionally left as informal in table examples only.
  2. "developer-workflow changes" category boundary — add a process-documentation carve-out or confirm the current boundary is intentional.

The two FOLLOWUP items (duplicated derivation rule, no transition note) are advisory and do not block merge.

`merge_authority: auto_merge_when_gates_pass` applies after pending CI checks complete and the two DISCUSS items are resolved or explicitly waived.

Comment thread .agents/workflows/post-merge-audit.md
Comment thread .agents/workflows/pr-processing.md Outdated
Comment thread .agents/skills/post-merge-audit/SKILL.md
@claude

claude Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Code Review

Overall: Solid, well-structured documentation PR. The QA lane contract is internally consistent across all five files, the capitalization convention (uppercase UNKNOWN vs lowercase unknown) is applied correctly throughout, and the seven QA Evidence block examples cover the key operational states. The in_progress phase-dependent semantics (no-action during active batch, readiness blocker at release) are adequately signaled both in the field template description (pr-processing.md:505) and in the post-merge audit classification section.

Two minor concerns:

1. Missing mixed-batch example. The Batch QA Lane section (pr-processing.md lines 462–467) describes mixed batches — applying QA only to the qualifying subset of a batch — but none of the seven QA Evidence examples demonstrates this case. In practice a mixed batch (e.g., three doc PRs plus one CI config PR) is probably more common than the all-or-nothing cases shown. An eighth example would clarify how Scope checked and Tested at should be scoped to the qualifying PRs only, and how the rationale explains the subset decision. Not a blocker for this PR, but worth considering as a follow-up.

2. Minor intra-file redundancy in pr-batch/SKILL.md. Lines 347–350 ("For every batch that requires QA under the canonical workflow's Batch QA Lane decision, make sure the QA lane is represented...") largely repeat what lines 150–160 already state in the same file. Two mentions of the same rule in close proximity within a single skill file could create confusion if they're ever updated out of sync. Consider trimming the later paragraph to a forward reference only, or merging both into one place.

Neither issue affects correctness of this PR's changes. Everything else — the release-blocking status derivation rule, the waiver verification requirements, the unknown-not-stalled design choice, and the coordinator closeout gate addition — looks correct and consistent.

Comment thread .agents/workflows/pr-processing.md
Comment thread .agents/skills/pr-batch/SKILL.md
@claude

claude Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Review: Batch QA Lane Documentation

The five-file change is coherent and the core QA lane concept is defined consistently — capitalization convention (`UNKNOWN` vs `unknown`) is stated correctly in every file, the Evidence block structure is sound, and the scope algorithm mirrors correctly. A few issues below that range from minor wording imprecision to a genuine staleness-definition gap worth closing before this pattern is relied on in production audits.


Issues found

1. Cross-reference imprecision — "section label" that isn't one (minor)

`plan-pr-batch/SKILL.md` line 75–76 and `pr-batch/SKILL.md` lines 95–97 and 153–154 both refer to the "Allowed fallback evidence:" label in `pr-processing.md` as if it is a section heading or named definition block. In `pr-processing.md` it is a single bold inline label inside a bullet (line 486), not a section header. A reader skimming for a section titled "Allowed fallback evidence:" will not find one. The cross-references should say "the bold label Allowed fallback evidence: within the Batch QA Lane section bullet on private-state fallback" (or simply quote the one-sentence rule directly).

2. `Tested at` third option not propagated (minor, but affects audit correctness)

The canonical QA Evidence block (`pr-processing.md` line 503) defines:
```
Tested at: <PR/head SHA(s), audited range, or "not applicable: no PR/code changes">
```
`post-merge-audit/SKILL.md` line 125–126 only mentions "the PR/head SHA or audited range" — it does not include the `not applicable` carve-out. An auditor mechanically following `post-merge-audit/SKILL.md` would flag a `not applicable` `Tested at` value as missing a SHA, even though the canonical block explicitly allows it. The audit skill should mirror the full three-option definition.

3. `QA lane` field is unstructured (design note)

`pr-processing.md` line 501 packs six distinct data points (agent id, branch/worktree, claim status, last heartbeat status, owner requirement, UNKNOWN caveat) into a single free-form string field. The `post-merge-audit` step must verify claim/heartbeat state from this field, but without a defined separator the values cannot be reliably parsed or validated. Consider either a structured sub-block or a defined delimiter (e.g., `|`) so audit verification can be mechanical rather than natural-language.

4. "Stale" QA evidence is never defined (gap)

Both `pr-processing.md` (line 791, 888) and `post-merge-audit/SKILL.md` (line 164) flag "stale" QA evidence as a readiness blocker, but no file defines what makes QA evidence stale (commits pushed after `Tested at` SHA? Force-push? Time window?). `post-merge-audit/SKILL.md` line 125–126 implicitly handles this by requiring `Tested at` to match the current PR head SHA, which is the right operationalization — but it should be stated as the definition so agents following the rule know why they're checking SHA parity, not just that they should.

5. `Batch Plan Format` "QA Lane decision" field has no template (minor)

`plan-pr-batch/SKILL.md` line 175 adds `Batch QA Lane decision and QA Evidence expectations:` to the Batch Plan Format but provides no example or template value. The Goal Prompt template (line 194+) does show: `Batch QA Lane: <required: lane/owner/scope/private-state or UNKNOWN fallback | not required: rationale>`. The Batch Plan field should either link to the Goal Prompt format or provide its own equivalent inline template, otherwise the two plan artifacts may diverge in content.

6. Field name casing inconsistency in Completed Batch Handoff Prompt (nit)

`post-merge-audit.md` line 87 uses "release-blocking status" (lowercase) in a checklist that the QA agent fills in, but the canonical QA Evidence block field is `Release-blocking status` (title case, line 510 of `pr-processing.md`). An agent filling out the handoff prompt may use inconsistent casing, making post-hoc extraction brittle.


No concerns with:

  • The `UNKNOWN` / `unknown` capitalization convention — stated precisely and consistently in all five files.
  • The scope-algorithm sync note and the mirroring rationale.
  • The worked examples in `pr-processing.md` (lines 527–636) — they cover the main states clearly.
  • The "not required" rationale requirement — properly blocks lazy QA omissions.

Comment thread .agents/skills/plan-pr-batch/SKILL.md
Comment thread .agents/skills/pr-batch/SKILL.md
Comment thread .agents/skills/post-merge-audit/SKILL.md
Comment thread .agents/workflows/pr-processing.md
Comment thread .agents/skills/plan-pr-batch/SKILL.md
Comment thread .agents/workflows/post-merge-audit.md
@justin808

Copy link
Copy Markdown
Member Author

Address-review summary

Scan scope: full review loop through 2026-06-23T09:07:40Z.

Mattered

  • Fixed the confirmed QA lane consistency issues through current head e48a4aa360d85bfce634c562d5ec798ff2faec36.
  • Replied to and resolved all current-head review threads.
  • Strict merge ledger passes with complete_allowed: true, no violations, and no unknown fields.

Optional

  • Advisory cleanup suggestions not needed for this PR were replied to with [auto-deferred] rationale and resolved.

Skipped

  • None.

Next default scan starts after this comment. Say check all reviews to rescan the full PR.

@justin808 justin808 added this pull request to the merge queue Jun 23, 2026
Merged via the queue into main with commit 67d63ca Jun 23, 2026
57 checks passed
@justin808 justin808 deleted the jg-codex/4127-batch-qa-lane branch June 23, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-hosted-ci Run optimized hosted GitHub CI for this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant